\chapter{Multi-Agent Modeling Framework}
\label{chap:multiagent}

\section{Agent Architecture Decomposition}

Hierarchy-aware agents implement perception, planning, execution, and verification loops.

\begin{definition}[Hierarchical agent]
An agent $i$ at level $\ell$ maintains:
\begin{itemize}
    \item \textbf{Perception buffer} $\mathcal{P}_i$ summarizing local observations and upstream directives.
    \item \textbf{Planning module} $\Pi_i$ selecting options (\cref{def:hpomdp}) using contextual bandits.
    \item \textbf{Execution controller} $\mathcal{C}_i$ dispatching low-level actions.
    \item \textbf{Verification monitor} $\mathcal{V}_i$ auditing compliance and logging metrics.
\end{itemize}
\end{definition}

\section{Tool and Option Selection}

We model tool selection as a contextual bandit problem with context $c_t \in \Context$, action (tool) $a_t \in \Tools$, and reward $r_t$.

\begin{assumption}[Linear bandit]
Expected reward $\E[r_t | c_t, a_t] = c_t^\top \theta_{a_t}$ with $\|\theta_a\| \leq S$.
\end{assumption}

\begin{theorem}[Regret bound under hierarchical feedback]
\label{thm:bandit_regret}
Using LinUCB with regularization $\lambda > 0$, learning rate tuned to feedback delay $d_\ell$, the cumulative regret after $T$ rounds satisfies
\begin{equation}
R_T = \mathcal{O}\big(d \sqrt{T \log(1 + T/\lambda)} + d_\ell \big),
\end{equation}
where $d$ is context dimension and $d_\ell$ accounts for delayed top-down signals.
\end{theorem}

\begin{proof}
Standard LinUCB analysis \cite{auer2002} extends by incorporating delayed feedback term $d_\ell$. Appendix~\ref{app:proofs} outlines the derivation.
\end{proof}

\section{Coordination and Communication}

Coordination matrices $C$ encode influence weights across agents.

\begin{definition}[Coordination spectrum]
Let $C \in \mathbb{R}^{n \times n}$ with entries $c_{ij} \geq 0$ and row sums one. The spectral radius $\rho(C)$ quantifies convergence speed of consensus updates $x_{t+1} = C x_t$.
\end{definition}

\begin{proposition}[Convergence with governance constraints]
If $C$ is doubly stochastic and $\rho(C - \frac{1}{n}\mathbf{1}\mathbf{1}^\top) < 1$, then consensus to the average occurs. Additional governance constraints encoded via projection $\Pi_{\mathcal{S}}$ ensure convergence within safe sets: $x_{t+1} = \Pi_{\mathcal{S}}(C x_t)$ remains bounded provided $\mathcal{S}$ is convex and closed.
\end{proposition}

Communication protocols include publish-subscribe overlays, gossip-based averaging, and structured broadcasts. Bandwidth budgets map to the information bottlenecks analyzed in Chapter~\ref{chap:information_theory}.

\section{Governance and Safety}

Safety invariants enforce guard conditions using temporal logic.

\begin{definition}[Safety specification]
A safety property $\varphi$ in linear temporal logic (LTL) constrains traces of system states. Examples include $\square (\text{queue	extunderscore length} < B)$ or $\square(\text{critical	extunderscore tool	extunderscore active} \rightarrow \lozenge_{\leq \tau} \text{supervisor	extunderscore ack})$.
\end{definition}

Agents implement runtime monitors evaluating shield functions $S_i(x_i, \Phi)$ that block unsafe actions and escalate to higher levels. Governance escalation thresholds align with risk registers described in Chapter~\ref{chap:introduction}.

\section{Implementation Patterns}

Appendix~\ref{app:algorithms} lists pseudocode for agent loops. Key patterns:
\begin{itemize}
    \item \textbf{Sense-plan-act with verification}: Sequence perception, option selection, action dispatch, and post-condition checks.
    \item \textbf{Hierarchical event handling}: Upstream alerts trigger policy revisions via option re-planning.
    \item \textbf{Traceability hooks}: Each action logs context, rule ID, outcome, and downstream impacts.
\end{itemize}

\section{Integration with Simulation Infrastructure}

Parameter registries described in Chapter~\ref{chap:simulation} include agent counts, communication topologies, and governance thresholds. Metrics tracked in Chapter~\ref{chap:experiments} (e.g., regret, latency, safety incidents) validate the effectiveness of the multi-agent design.

